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[57] ABSTRACT 

A computer implemented application development 
(authoring) system permits objects (such as VBX custom 
controls) to be graphically inserted into the program under 
development by dragging and dropping associated icons into 
one of four views. The properties associated with the object 
may then be assigned settings. Development of a complete 
application is accomplished by visually arranging, ordering, 
and interconnecting the objects without the necessity of 
writing any code. The four views of Output, Map, 
Multitrack, and Workform may be synchronized so that 
changes made to the program in one view are simulta- 
neously reflected in all other views. The system generates as 
output a script listing the objects and their properties which 
is then executed by a separate run time program. The system 
permits use of objects written to a standard specification and 
the addition at any time of additional objects written to mat. 
Integration of the objects into the system is achieved by 
wrapping each object in an "envelope" of system specific 
properties. 

4 Claims, 6 Drawing Sheets 
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VISUALLY ORIENTED COMPUTER 
IMPLEMENTED APPLICATION 
DEVELOPMENT SYSTEM UTILIZING 
STANDARDIZED OBJECTS AND MULTIPLE 
VIEWS 

FIELD OF THE INVENTION 

This invention relates to the use of a computer imple- 
mented application development process, and, more 
specifically, to an authoring system and its associated run 
time system which permits the creation of an application 
(program) in which standardized objects are incorporated 
into the application without the necessity of the user having 
knowledge any programming language or programming 
script, and in which the application development process can 
be monitored (observed, evaluated, etc.) through the mul- 
tiple synchronized views of output display, timing sequence, 
logical relationships, and script workform. 

BACKGROUND OF THE INVENTION 

The last ten years has seen an explosive growth in the 
development of digital computer systems and technology 
both in the processing power and of the hardware systems 
and in the sophistication of the controlling software. Not 
only has the microprocessor become a ubiquitous incident to 
almost every laboratory, industrial, and household 
appliance, but many functions once assigned to combina- 
tions of moving parts are almost invariably now preformed 
by digitally controlled electronics. Systems as diverse as 
automobile ignition systems and home turntables now share 
a common electronic heritage. In fact, the performance of 
many electronic circuits can be equally implemented by 
hardware or by a digital computer under software control. In 
many instances the software component of the digital elec- 
tronics has become enormously complex and correspond- 
ingly expensive to develop. Indeed, the cost of the electronic 
hardware has come down so far that the development time 
and cost of the software is now the limiting element in the 
introduction and pricing of many new systems. There has 
developed, therefore, a great need to reduce the cost and 
time of software development. 

The development over the past three years of what is 
referred to as "multimedia" computer systems has increased 
the complexity of the required software as the systems 
access, simultaneously present, and then replace vast 
amounts of information. Present systems are directed toward 
the simultaneous presentation of audio and visual output, but 
systems are clearly on the horizon which will present 
coordinated output which can be sensed by all five human 
senses and create a virtual reality. While present systems 
access the information for multimedia presentation prima- 
rily from local magnetic and optical storage media, it is 
foreseeable that electronic and optical communication with 
the world through the coming "information superhighway" 
will provide a rich source of interactive multimedia 

In an attempt to reduce the cost and complexity of 
developing sophisticated software, attempts have been made 
over the last few years to modularize software functions 
with the idea that such modularized software could then be 
integrated into larger systems to achieve more complex 
functions. Each piece (object) of modularized software 
would consist of a set of instructions designed to perform a 
specific type of task which was frequently required in a more 
complex program. To use the objects, no knowledge of their 
internal coding structure would be necessary. All that would 
be required is to provide the modules with the information 
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(parameters) they required for their use. Thus, applications 
would be built by selecting appropriate software modules 
and assembling/ordering/sequencing them to achieve higher 
order functions. This is the essential goal of "object oriented 

S programming". It quickly became apparent that building 
with modularized software would not be efficient unless the 
presentation of the modules themselves conformed to some 
form of standard; i.e., flat bricks of any material can be 
stacked, mixed flat and curved bricks, no matter what 

10 material they are made of, do not stack well. A second 
problem facing object oriented programming is that effi- 
ciencies of application construction would only be achieved 
when the assembly of the objects did not require extensive 
and complex programming itself. 

15 Several attempts have been made, therefore, to develop 
visually oriented programming systems in which software 
modules are manipulated as icons which may be arranged/ 
ordered as needed and interconnected with logical operators. 
Examples of this can be found in the NeXT application 

20 development system, in the VIP Macintosh program, and in 
PowerBuilder, a Microsoft Windows program. Generally, in 
these types of application development systems, the soft- 
ware modules or objects are presented on a palette from 
which they are selected and "dragged" into the application 

25 under development. Ideally, in these systems, the software 
instructions needed to integrate the objects into a function- 
ing whole (corresponding to the visual representation or 
map) are provided by the authoring program system. In 
reality, most of the prior art systems require the user to write 

30 some of the interconnecting software code either in a 
common higher order language or in a language specialized 
to that program. 

It was hoped that the demand for complex and varied 
software imposed by multimedia programming require- 

35 ments could be efficiently addressed by the use of object 
oriented programming. Until the present invention, these 
hopes have not been realized. A major problem not solved by 
the prior art has been the inability to handle in one visual, 
non-scripted, application development system the diverse 

40 requirements of different types of data required by a multi- 
media program in appropriate visual interfaces. 

For instance, during the authoring process a view of the 
output generated by the application under development is 
useful when dealing with visual output (images). However, 

45 such a view is much less useful for dealing with audio output 
since, clearly, audio can not be seen. Correspondingly, a time 
line view in which the elements of the application program 
are represented by graphs plotting the relative times and 
sequences of presentation is much more useful for audio. 

50 Similarly, a logical map (flowchart paradigm), which visu- 
ally relates the sequence and logical interconnections of 
each program element (visual, audio, etc.) is very useful for 
designing and following the basic structure of the applica- 
tion program. Sometimes, for quickly manipulating or edit- 

55 ing particular parts of the developed application code, it is 
more useful to be able to view the program in a text view and 
treat the underlying script like a text document. Not only 
should the application development system provide, at any 
given moment, a view appropriate to the type of multimedia 

60 material being utilized, but such views should be able to be 
synchronized so that changes made in one view are instantly 
reflected in all the others. Prior art application development 
programs do not have the flexibility to present all four views, 
much less in a synchronized manner. Programs are available 

65 which are optimized for output view, map view, timeline 
view, and script view. Some permit more than one view with 
some synchronization. However, no prior art program per- 



05/05/2004, EAST Version: 1.4.1 



5,862,372 

3 4 

mits access to all four synchronized views. In particular, no FIG. 3 shows a Multitrack view of a simple program, 
prior art program has successfully overcome the difficulty of FIG. 4 shows a Workform view of a simple program, 
simultaneously synchronizing the time line view to the other FIG. 5 shows for a simple program all four views, Output, 
V1CWS * Map, Multitrack, and Workform simultaneously displayed in 

Additionally, in an optimum visual, object oriented, appli- 5 separate windows, 
cation development program there should be available a nG 6 shows a t M the objects 

wide variety of objects (software modules that would which m ava]lablc tQ ^ ^ of ^ tem 
manipulate m different manners multimedia output) with the 

ability to easily add additional objects, developed by inde- DETAILED DESCRIPTION OF THE 

pendent programmers, to the development system. Each 10 INVENTION 
added object should seemlessly be integrated so that it is 

available in all four views. Programs of the prior art each Definitions 

have their own object definition standards. That is, for an T , . . , . . a . . , . 

. . ... . i • . .v • . v * 1° order to avoid the confusion often present in the world 

obiect to be incorporated into their system, it must meet that f . f r 4 f. v 4 - 4 . 

J , j j xt • J • -i i_t k of computer jargon, for purposes of this application, the 

program s standards. No prior art program is available 15 c „ • ( L !i l *l * J _*u u i 

* . 1t « , r r i • j . , A following terms shall have the meanings set forth below: 

which will allow addition of objects not designed to its ° ° 

standard without the necessity of writing additional code to "APPLICATION" shall mean an end product computer 
perform the interface to that object. Finally, it should be program which is ready for use by its intended audience, 
noted that application development systems necessarily con- "CODE" shall mean a series of specialized words and 

sist of two major parts. Hie first is the authoring section 20 symbols in a particular syntax which must be entered 
which provides a customized output (script). The second is through a keyboard and which is then converted to a 
a run time system which takes the customized output and machine readable form. 

executes it. Some prior art run time systems for visually "OBJECT" shall mean a modular software program writ- 
developed applications are notoriously slow in their execu- ten to a defined specification which may perform any 

tion, 25 function, For purposes of this application, the word 

"OBJECT* shall be considered synonymous with the term 
SUMMARY OF THE INVENTION "custom control". Objects are intended to be grouped and 

The present invention overcomes the limitations of the assembled into applications, 
prior art. First, the system can be used to author applications 3Q "SCRIPT" shall mean a set of computer instructions 
using an entirely visual programming scheme (paradigm) written at a higher level than code i.e., farther removed from 
which does not require the user to know or be able to write machine language, which is usually interpreted during 
any specialized code. Icons representing the objects (and execution. "SCRIPT" for purposes of this application shall 
accordingly their functionalities) may be placed (dragged) also refer to the listing of objects and property settings 
into an appropriate view. The system of the invention 35 generated by the authoring program of this invention, 
generates the information needed by the run time program. "SYSTEM" shall mean the computer implemented appli- 
Second, during authoring, the user can access all four views cation development system of this invention, 
which can be synchronized or not at the user's choice, and "WINDOWS" (when appearing in all capital letters) shall 
these views are available in overlapping windows. The user mean the Microsoft operating system of the same name 
may, thus, work in the view most appropriate to the type of 4Q without regard to the version number, 
object the user is manipulating. Third, the objects which are present invention is an integrated computer imple- 
available on the palette, can be added or subtracted from the mC nted application development system which permits the 
development program with great ease. While the present ^ to create custom applications without writing a line of 
implementation of the invention is designed to directly computer code. The ability to do just this has existed as a 
operate standardized Microsoft Visual Basic VBX controls 45 compu ters moved into the public imagination 
as objects, the manner in which the invention wraps the md thc dream is f^ed by this invention. The invention 
objects is equally usable with objects written to standards provides a fully visual programming environment in which 
other than the Microsoft VBX control. Fourth, the invention ±c ^ can create WINDOWS applications in 
is capable of utilizing any objects wntten to a standard hours ^ t would take days or weeks using sta ndard pro- 
specification. That is, at its core it is featureless and exten- 5Q grammmg languages. When describing the process imple- 
sible. The invention imposes no limitation on the type of m ented by a non-authoring computer system, it is generally 
function the objects can preform, and the objects need not be sufficient to describe the process steps carried out and their 
directed towards multimedia or any other single use. Thus, order However, authoring programs create an environment 
all types of programs can be developed utilizing the entire ^ many potentialities within which the user works to 
range of available objects, and, during application $s develop me application. To describe authoring programs it is 
development, the objects may all be presented in the four necessary to describe the environment. Consider the analogy 
views, once again synchronized or not. Fourth, a run time t0 buMng a housc ^ cnvironmcnt providcs the t0 ols 
program is provided which executes the output of the (hammers, nails, saws, lumber, wires, pipes, etc.) as well as 
authoring program much faster than the runtime programs of ways of looking at (hc house mdet instruction 
the pnor art. These, and other advantages of the invention 6Q (architectural plans, outside drawings, kitchen layouts, etc), 
will become apparent from the description of the invention 0ncc me cnv i ronmen t contains all the tools and views 
which follows. needed, the user can construct the house according to his/her 
BRIEF DESCRIPTION OF THE FIGURES own needs by assembling the parts. 

This description of the computer implemented application 

FIG. 1 shows a schematic representation of the major 6S development system will briefly: 1) describe the environ- 
components of a modern programmable digital computer. me nt created by the present invention; 2) describe the 

FIG. 2 shows a Map view of a simple program. process of using the system to create applications; 3) 
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describe the manner in which the system creates a script 
corresponding to the program elements and how that script 
is used by the run time program; and 4) describe how the 
system coordinates objects and permits additional objects to 
be added easily. A complete description of the invention is 5 
contained in the source code provided in the software 
appendix which is attached to this application. 

FIG. 1 is a block diagram showing the basic functional 
units of a modem digital computer with which the computer 
implemented applications development system of the inven- 10 
tion works. Information may be stored on magnetic or 
optical media for access or be available through a connec- 
tion to a remote information source. During the development 
of an application with the system of this invention, the user 
directs the incorporation of objects into one of the four 15 
views, modifies the properties associated with that object, 
and interconnects the objects so chosen. The user may direct 
the machine to accomplish these tasks using a pointer (such 
as a mouse) or a keyboard and visually observes his/her 
actions on the system's output display. As the system 20 
responds to the user's directions, it creates a script repre- 
senting the application being developed by the user. When 
the development process is complete, the user stores the 
script on one of the system's storage devices. 

One of- the- interesting- features of the object oriented 3 s 
authoring system of this invention is that no traditional 
looking code, written in a programming language, is gen- 
erated by the system. Rather, the output of the authoring 
system is a listing of the objects in order of appearance (in 
the program sequence) along with the properties associated 30 
with that object at that place in the sequence. The properties 
are the information or settings which specify to the object 
how it will perform. The computer implemented application 
development system records the settings for the objects and 
the order of execution of the objects in an output script. 35 
Thus, the system operates only at the object level. As the 
author proceeds in developing the application, the system 
adds to the script. 

The four views are different ways to make and look at the 4Q 
same script. Each has its own special perspective and 
strength. FIG. 2 shows a Map view of a simple program in 
which the flow of the program and the flow of data is 
displayed. The Map view displays a block diagram in which 
the authored program is viewed as the sequence in which the 45 
objects are utilized, the branches which are taken, and the 
flow of data between the objects. The system not only allows 
the passing of data back and forth between different objects, 
but allows objects to affect each other in other ways as well. 
The Map view is best for seeing and building the program 5Q 
using a flowchart metaphor. Thinking of the program in this 
way can be very useful when building interactive applica- 
tions since this view presents a way to see the flow of data 
between objects which can save hours of debugging time. 
When a Map window is opened, the system generates the S5 
Map view directly from the underlying script. 

The Output view displays the screen output of the objects 
and is best for working directly with the product of the 
script. In this view you see the output of each object you 
incorporate into the program. When an Output view window 60 
is opened, the system generates the Output view directly 
from the underlying script. 

FIG. 3 shows a Multitrack view of a simple program in 
which the timing sequences of object presentation are dis- 
played. The Multitrack view is limited to coordinating and 65 
timing the appearance, disappearance, and activities of the 
objects. It is different from the other views because it is not 
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a view of just any part of the script (program), it only views 
timed sections. This view is useful for creating parts of the 
application that require objects to be created and destroyed 
at specific times. The Multitrack view automatically sets all 
the special properties that differentiate the objects in these 
time sections form the objects in other parts of the applica- 
tion (script). 

FIG. 4 shows a Workform view of a simple program 
where each object and its associated properties is displayed 
in each row. The Workform view displays the underlying 
script directly and permits treating it like a text document. 
The script can be viewed in as little or as much detail as is 
necessary. This view has word processing features so that it 
becomes easy to make global changes, and to copy and 
move blocks of information. 

FIG. 5 shows for a simple program all four views, Output, 
Map, Multitrack, and Workform simultaneously displayed in 
separate windows. In each view, the system of this invention 
permits development of the application using the same 
method of adding an object, interconnecting the object, and 
setting the objects properties. For instance, from a palette of 
objects, which displays a collection of icons that represent 
each object available to the system (such as the palette 
shown in FIG. 6), the user of the system drags an object into 
the view. The system simultaneously generates the under- 
lying script which reflects the object and the properties 
associated with the object. The user's double click on the 
icon representing the object presents a menu list of the 
properties for that object the settings of which can be 
changed according to the user's requirements. If a separate 
dialog box is available for a particular property, clicking on 
the property gets the user to the dialog box. As a property 
setting is defined, the system records that definition in the 
associated underlying script. In addition, the system user can 
elect to have all the four views synchronized so that each 
view is updated for every change made in any view to the 
underlying script. 

For example, in the Output view window the user might 
drag an icon representing an object which pulls a picture into 
the application program. The Output view window would 
then display the picture. If several pictures were to be 
presented in some order, several picture icons would be 
placed in the output view window. The user could set the 
order of presentation in the Workform view, but might more 
graphically do so in the Map view. In the Map view the user 
could arrange the picture icons in the order in which he/she 
wished the pictures to appear. In addition, in the Map view, 
the user could add conditional branches from picture to 
picture. The display timing for each picture could be set 
through the properties boxes or the Workform view, but 
might more graphically be done in the Multitrack view 
where each pictures duration and sequence would be indi- 
cated on parallel time lines. Finally, if the user wished to 
duplicate part of the application already created, he/she 
could use the Workform view to copy and move the under- 
lying script. Clearly, the user of the system has the option of 
choosing the view most appropriate to the type of applica- 
tion under development. Since the system is a general 
purpose programming environment, the user will choose the 
view most appropriate for the application. 

It should be noted that a very powerful feature of the Map 
view resides in its ability to permit the application developer 
to specify that one object in the sequence get or set data 
from/to another object. Thus, information exchange among 
objects is enabled as well as logical branching and program 
flow between objects. 

In order to run a program developed in this manner from 
the script created by the authoring system, a run^ time 
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program is required which executes each object in the 
defined sequence with the defined property settings. The 
system of this invention provides such a run time program. 
Programs authored by the computer implemented applica- 
tion development system of this invention run faster under 5 
its run time program than to object oriented programs 
developed and run under Microsoft Visual Basic. 

The computer implemented application development sys- 
tem of this invention is able to integrate and work with the 
objects in the above described manner because, in addition 10 
to the internal properties which are unique to each object and 
relate to the special work that each object does, the system 
wraps all objects in a kind of "envelope" of special prop- 
erties that regulate how the objects act in the system. These 
special properties are: 35 



20 
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Comment 
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Type 


Destroy 


WaitFor 


Destroy Type 


Group 



No other program in the prior art wraps standardized objects 25 
in this manner to achieve similar functionalities. This wrap- 
ping is also what permits the easy integration of additional 
objects into the system. Also, the system is so designed that 
the number of such special properties may be increased, if 
necessary, to adapt to any changes in future definition of 30 
objects. 

As noted earlier, there are many different standards for the 
format of objects which have been devised by the propo- 
nents of object oriented programming. Many prior art 
authoring systems use objects with their own proprietary 35 
standard. However the implementation of the authoring 
system of this invention disclosed in the software appendix 
has been designed to operate with "VBX custom controls" 
(objects) written to the Microsoft Visual Basic and Visual 
C++ standard. Presently, there are about 300 software ven- 40 
dors across the United States developing and selling such 
objects. Thus, a large number of objects written to the same 
standard are now available. The Microsoft Visual Control 
Pack contains representative objects and examples of code 
for different control types. Any object written to this stan- 45 
dard may be used with the system of this invention. 
However, the methods employed by the present invention to 
wrap objects and integrate and run them directly from the 
system are equally applicable to objects written to other 
standards, and the scope of the present invention should not 50 
be considered limited by reference to the VBX control type 
object. 

The source code in the software appendix does not 
contain any objects. As noted above, objects may be 
obtained through a large number of commercial suppliers. 55 
However, the system of this invention is sufficiently general 
to permit development of applications of all types. Special- 
ized applications may be developed by importing appropri- 
ate standardized objects into the system. Thus, the system is 
infinitely extensible since it provides the necessary back- 60 
bone environment for application development with objects 
written to a common standard. 

While the software appendix discloses the source code for 
the computer implemented application development system 
of this invention in the WINDOWS and IBM-PC platform, . 65 
the invention is not platform dependant and could also be 
implemented on the Macintosh and other platforms. In order 



to run the source code disclosure of this invention on a 
digital computer, the following commercially available soft- 
ware programs should be available on the computer: 
Microsoft Visual C++ 1.5, Microsoft WINDOWS 3.1, and 
objects of the user's choice written to the Microsoft VBX 
Custom Control standard, Version 1 .0. For Microsoft Visual 
C++ 1.5, file VBAP1.H has been renamed to VB_VBAPI.H. 

The computer implemented application development sys- 
tem of this invention is presently available from 
V__GRAPH, Inc. under the trademark OZONE. 

What is claimed is: 

1. A computer implemented application development sys- 
tem for visually developing an application without the 
necessity of writing any code comprising: 

a. means for wrapping objects written to a standard 
specification; 

b. means for establishing four views; 

c. means to synchronize said views; 

d. means for moving an object into one of four said views; 

e. means for setting the properties of said object; 

f. means for moving and setting the properties of addi- 
tional objects until all of the desired objects have been 
specified; 

g. means for interconnecting the objects in temporal 
sequences; _ 

h. means for specifying the flow of data and control 
between the objects; 

i. means for generating a script reflecting means C. 
through H.; and 

j. means for running said script. 

2. The system of claim 1 in which the objects written to 
a standard specification are VBX Custom Controls. 

3. A computer implemented application development sys- 
tem comprising: 

a. means for storing objects written to a standard speci- 
fication; 

b. wrapper means for encapsulating said stored objects; 

c. means for visually displaying the application under 
development comprising: 

(1) means for displaying an output view; 

(2) means for displaying a map view; 

(3) means for displaying a multitrack view; and 

(4) means for displaying a workform view, 

d. means for providing synchronization among four said 
visual display means; 

e. means for selecting from said object storage means one 
or more of said wrapped objects for inclusion into the 
application under development; 

f. means for setting the properties of each selected 
wrapped object; 

g. means for setting the desired temporal relationships 
among said wrapped objects; 

h. means for specifying the flow of data and control 
among said selected wrapped objects; 

i. means for creating and storing a persistent representa- 
tion of said selected wrapped objects, the property 
settings of said objects, the temporal relationships 
among said objects, and the flow of data and control 
among said objects; and 

j. means for executing said representation. 

4. The system of claim 3 in which the objects written to 
a standard specification are VBX Custom Controls. 
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